Assessing the policy utility of routine mortality statistics: a global classification of countries

Abstract Objective To evaluate the utility and quality of death registration data across countries. Methods We compiled routine death and cause of death statistics data from 2015–2019 from national authorities. We estimated completeness of death registration using the Adair-Lopez empirical method. The quality of cause of death data was assessed by evaluating the assignment of usable causes of death among people younger than 80 years. We grouped data into nine policy utility categories based on data availability, registration completeness and diagnostic precision. Findings Of an estimated 55 million global deaths in 2019, 70% of deaths were registered across 156 countries, but only 52% had medically certified causes and 42% of deaths were assigned a usable cause. In 54 countries, which are mostly high-income, there is complete and high-quality mortality data. In a further 29 countries, located across different regions, death registration is complete, but cause of death data quality remains suboptimal. Additionally, 37 countries possess functional death registration systems with cause of death data of poor to moderate quality. In 30 countries, death registration ranges from limited to nascent completeness, accompanied by poor or unavailable cause of death data. Furthermore, 38 countries lack accessible data altogether. Conclusion By implementing more proactive death notification processes, expanding the use of digitized data collection platforms, streamlining data compilation procedures and improving data quality assessment, governments could enhance the policy utility of mortality data. Encouraging the routine application of automated verbal autopsy methods is crucial for accurately determining the causes of deaths occurring at home.


Introduction
Health planning and policy evaluation requires reliable, detailed and timely mortality statistics and cause of death data.Functional civil registration and vital statistics systems ensure the continuous registration and certification of all deaths in a population. 1 However, in many countries, the statistics on deaths and causes of death produced by civil registration and vital statistics systems are inaccurate and otherwise of limited use for public policy. 2Even when proper registration and certification of deaths occur, the information is not always compiled into routine statistics to guide government policy.Gaps in mortality data are often filled by surveys and censuses despite the fact that they are not a timely and routine source of mortality statistics. 3In many low-and middle-income countries cause of death data are only collected for the small proportion of deaths that occur in hospitals.Moreover, the diagnostic accuracy and policy utility of such statistics are limited by insufficient training in correct medical certification procedures and/or incorrect coding practices. 4Approximately 60% of deaths occur outside of hospitals in low-and middleincome countries. 5Data on these deaths can be unreliable because the cause is often ascertained by non-medical, untrained personnel, with or without the use of validated verbal autopsy methods.
Civil registration and vital statistics are designed to generate continuous, disaggregated, timely and accurate data on deaths and causes of death that can be used as a source of health intelligence for governments.Mortality data that is categorized by both age and gender can be used to compute essential population health metrics like life expectancy, and to monitor a nation's progress through the epidemiological transition.Reliable measurements of excess mortality during pandemics or from mortality shocks such as natural disasters or civil conflict are critical for decision-making.In addition, civil registration and vital statistics systems facilitate calculation of mortality indicators for subnational populations; in turn enabling a more granular understanding of differentials in mortality risk.Yet, despite these multiple uses, death registration data are often underused largely because of concerns about their accuracy and potential biases. 6imilarly, reliable and timely statistics on causes of death are fundamental for tracking trends in the leading causes of premature mortality; for underpinning assessments of the impact of health intervention strategies and policies; and are a prerequisite for monitoring progress for seven out of the 17 sustainable development goals. 7,8revious global assessments of the availability and quality of mortality statistics generated by civil registration and vital statistics systems are limited to a single source, such as the World Health Organization (WHO) Mortality Database, and rely on complex transformations of the underlying data; or are now several years out of date. 1,2,9,10 recent assessment of mortality data produced by the Global Burden of Disease (GBD) Study focused on the utility of vital registration for measuring national and sub-national burden of disease; however they failed to identify common challenges in countries at different levels of development of statistical systems, and often censored data points judged to be outliers, introducing bias into the results. 11ere we present a systematic evaluation of the completeness and quality of death and cause of death statistics for all publicly available sources of data where deaths were either registered or otherwise known to a government.We then use these results to classify country vital registration systems into categories that represent their utility for guiding public policy in countries.Based on these scores, we then propose a prioritized set of feasible, tested system interventions that can assist countries to derive maximum policy benefit from their administrative data systems.

Methods
We used death statistics generated from routine death registration or reporting systems designed to count all events within a specific jurisdiction, including hospital death reporting systems.We define these statistics as registered or reported deaths known to some level of government authority; hereafter we refer to them as registered deaths.Data sources were identified from an exhaustive search, and were only considered if they were compiled by national authorities and publicly available either through national reports (e.g.vital statistics reports); international databases which include only statistics notified by national authorities; national civil registration and vital statistics system assessments endorsed by governments; or data otherwise made available to the authors by relevant government agencies. 10,12,13These sources exclude mortality and cause of death statistics collected from Sample Registration Systems; Health and Demographic Surveillance System sites; and other surveys and censuses where the primary intent was not formal death registration.We used the WHO Mortality Database for cause of death data for the majority of countries, and country-provided data in some instances. 10A full list of data sources is provided in the online repository. 14We only considered data from 2015-2019 as indicative of an operational vital registration system, and we did not consider timeliness any further.
We evaluated the policy utility of vital registration data, based on the completeness of death registration and quality of cause of death data.Completeness was calculated for registered or reported deaths as the percentage of the estimated total number of deaths, measured where possible by year of occurrence using the Adair-Lopez empirical completeness method. 15This method estimates completeness of death registration using a statistical model with covariates of the registered crude death rate, and incorporates variables representing the drivers of the true crude death rate such as: (i) the under-five mortality rate to represent mortality level; (ii) the percentage of the population aged 65 years or older to represent population age structure; (iii) the completeness of under-five death registration or reporting; (iv) calendar year; and (v) country random effects. 15Such a method also ensures comparability by applying a common approach to calculating completeness for all data.
For countries where there is either high human immunodeficiency virus mortality or conflict-driven mortality, we used the United Nations (UN) World Population Prospects 16 estimated deaths to calculate completeness.The empirical completeness method is less reliable in such settings given the assumptions of the model; these countries are indicated in the online repository. 14UN estimates are predominantly derived from adult mortality calculated from registered deaths adjusted for incompleteness, and child and adult mortality estimates from censuses and surveys, which are entered into model life tables. 16We present completeness estimates in bands of five percentage points to account for uncertainty in estimation.We estimated global completeness of death registration by using the most recent completeness estimate for each country based on available data for the period 2015-2019; as applied to UN Population Division estimates of deaths in 2019. 16e included all cause of death data where the underlying cause was ascertained from the International certificate of medical cause of death (medically certified cause of death), subsequently coded according to the rules and procedures of the International statistical classification of diseases and related health problems, 10th revision (ICD-10), 2016 version. 17Garbage codes were classified into four different levels of severity (very high, high, medium and low) according to their potential effect on policy deliberations (online repository). 14,18Using this classification, we calculated the percentage of deaths with a useable cause; that is, the percentage of estimated total deaths that have a medically certified cause of death and for which the assigned cause was not classified as a garbage code of very high, high or medium severity.
Countries with a high proportion of deaths at older ages (older than 80 years) tend to have a higher proportion of garbage codes in their cause of death data because accurately diagnosing underlying cause of death at older ages is increasingly difficult due to higher likelihood of multiple morbidities.We therefore only considered cause of death data for those individuals younger than 80 years, to ascertain the extent of garbage codes in the data.
For 46 countries the distribution of garbage codes by severity could not be calculated because ICD-10 data were not publicly available.For these, we predicted the useable fraction of causes of death for individuals younger than 80 years based on a linear regression model using death registration completeness; GBD super-region; sociodemographic index; and universal health coverage as covariates (online repository). 14Usability calculations and estimates are also presented in bands of five percentage points to account for uncertainty in the completeness estimation method.Estimating the fraction of useable causes for these 46 countries enabled their inclusion in further analyses of civil registration and vital statistics system performance.We did not include country-level estimated usability percentages in global or regional calculations of usability.
To better target interventions and identify challenges in mortality statistics, the usability of countries' data was classified by the completeness of death registration and the quality of cause of death data, determined by the percentage of deaths in individuals younger than 80 years with a usable code.The criteria for categorizing mortality data from national vital registration systems are displayed in Table 1.Countries with 95%-100% death registration are labelled as complete since the small percentage of unrecorded deaths would unlikely influence policy decisions.Those with 75%-94% registration are termed functional; they record enough Bull World Health Organ 2023;101:777-785| doi: http://dx.doi.org/10.2471/BLT.22.289036   Policy utility of routine mortality statistics Tim Adair et al.
deaths to demonstrate that the system is operational, but many deaths go unrecorded.Countries with 25%-74% registration have limited systems.Despite being operational, a significant portion of deaths remain unrecorded, reducing the policy value of the data.Nascent registration indicates very few recorded deaths, highlighting a need to enhance basic system components.High-quality cause of death data means 80% or more deaths in people younger than 80 years have a usable cause, marking them as fitfor-purpose.Data with 60%-79% usable causes are moderate quality, suitable for policy discussions but requiring caution.Data with less than 60% usable causes are of poor quality.In many such countries, non-medical staff often determine causes, sometimes using verbal autopsy.While these classifications and thresholds might seem arbitrary, they offer a foundation to prioritize interventions for enhancing the usefulness of regular mortality data.

Results
Of the more than 55 million deaths estimated to have occurred worldwide in 2019, 70% were captured and reported by civil registration and vital statistics systems in 156 countries.Just over half (52%) had a medically certified cause of death, while only 42% of deaths in people younger than 80 years were assigned a useable cause of death; the remainder either being attributed to a garbage cause of high, medium or low severity; or where the medically certified cause of death contained insufficient detail to assess quality (Table 2).The African Region only registers a small minority of deaths (15% of estimated total deaths in that region) and even less with a medically certified cause of death (8%).Eastern Mediterranean (50%) and South-East Asia Regions (69%) each have moderate levels of death registration, but far lower medically certified cause of death completeness (28% and 17%, respectively) and cause of death usability (15% and 3%, respectively).Other regions have higher death registration completeness, with cause of death usability being highest in the European Region at 82%.
The distribution of all 194 WHO Member States according to the two primary measures of data usability is summarized in Table 3 and presented by country in Fig. 1 and in the online re-pository. 14More than half (83) of the 156 reporting countries had death registration levels of at least 95%; and of these, about 54 had systems that provided a useable cause of death for at least 80% of deaths among people younger than 80 years.Death registration systems in these 54 countries can be considered as complete, and of high quality, accounting for about one-quarter of global deaths, with most of them located in the Region of the Americas, the European Region and the five-high income countries of the Western Pacific Region (Fig. 1).A further 29 geographically diverse countries have systems that suc-cessfully capture all or most deaths; but only 60%-80% of deaths that occur are assigned a useable cause in 20 of these countries, falling to less than 60% in a further nine countries.Of the remainder, more than half (43 countries) had systems that were at least 75% complete, and in some cases close to 95% complete; but where cause of death data quality was assessed as being predominantly moderate (20 countries) or poor (17 countries). 18 geographically diverse group of 21 countries account for about one tenth of global deaths and have vital registration systems that provide lim- ited policy benefit; typically capturing between 25%-74% of deaths but with useable fractions well below 60%.We identified data for nine countries where data completeness and quality are so low that we have classified the data systems as nascent.For 38 countries, we could find no vital registration data on mortality, or the most recent data was from before 2015.Likely, these 38 countries have some form of mortality reporting system established and operated by a government; however, the data were unidentifiable using our search strategy.

Discussion
In 2019, 70% of global deaths were reported to a government agency, suggesting a potential improvement from the 59% global death registration rate indicated by the GBD Study in 2015. 19owever, less than half of deaths among individuals younger than 80 years received a cause of death usable for public health planning.While nearly half of WHO Member States provided comprehensive mortality data, in 38 countries no data could be identified from 2015 to 2019.We found that for approximately half of deaths, the cause of death is not determined from a medically certified cause of death, and when they are, a significant portion either gets attributed to a cause of no public health utility or lacks specific details.Addressing this problem requires proper physician training to ensure accurate death certification.Proper cause of death determination would also have a positive impact on reducing misdiagnoses among deaths younger than 80 years, where the cause is useable but not necessarily correct.][22] We classified countries into nine categories of data usability based on a joint assessment of completeness and quality, quantified by the fraction of garbage codes found in the data.Countries that have complete registration also tend to have higher quality cause of death data.These were mostly high-income countries, and civil registration and vital statistics improvement strategies in these populations should focus on improving diagnostic practices among physicians to reduce garbage coding, especially for deaths in individuals younger than 80 years.Most countries with functional, but still incomplete registration also have substantial challenges with cause of death data quality.Countries in this category tended to be more geographically diverse.In countries where mortality data is nascent or exhibits low completeness and poor certification quality, enhancing vital statistics systems should be a priority.These efforts should include capacity-building and training in data analysis and dissemination as key strategies for reliably monitoring progress of forthcoming global health initiatives.
The coronavirus disease 2019 (CO-VID-19) pandemic highlighted the poor state of death registration systems in many countries, with most low-and middle-income countries being unable to generate timely and reliable data on excess mortality during the pandemic.Globally, only 73 of 194 countries produced full national data in 2020 and 2021 to enable calculation of excess mortality; most of them high-income countries. 23Measuring excess mortality reliably becomes challenging when registration is incomplete; distinguishing actual mortality increases from the pandemic's effects on the death registration system is a complex task. 24,25In many countries during the pandemic, death registration systems were unable to function as normal due to the impact of movement restrictions, a reliance on paper-based registration, and as a consequence of COVID-induced mortality. 26][29] Improving civil registration and vital statistics systems is a significant, organizational, technical and financial challenge for many countries.Countries seeking to improve their civil registration and vital statistics systems can benefit from the WHO Civil registration and vital statistics strategic implementation plan 2021-2025.This comprehensive platform enables countries to prioritize interventions tailored to their needs, emphasizing strengthened intersectoral collaboration and governance for vital statistics. 30This plan offers many feasible options for improving death registration and diagnosis based on recent methodological research and experience with practical implementation in countries.Specifically, this cumulated experience suggests that governments can work to improve the completeness of registered deaths by focusing on more active death notification processes for deaths occurring outside hospitals; on innovative application of digitized data collection platforms; on improving the national compilation of data; and on implementing processes to assess data quality. 6Concurrently, there should be accelerated efforts to increase use of the International medical certificate of cause of death (with appropriate training of physicians to reduce deaths with a garbage cause), and to fill the cause of death data gap for non-hospital deaths with routine, validated automated verbal autopsy methods. 21,31Training of physicians, including in medical school curricula, should be a priority in countries with complete registration but more than 20% of garbage codes, using the effective training methods and tools available. 21,22The introduction of ICD-11 will present additional challenges for its implementation into death registration systems; reinforcing the need for accelerated efforts to train doctors in correct medical certification procedures for deaths; and coders in the application of ICD-11 coding rules.
Monitoring the impact of civil registration and vital systems interventions through periodic assessment of data completeness and quality can now easily be performed for subnational populations using established data quality assessment software, such as ANACON-DA (co-developed by the University of Melbourne, Melbourne, Australia and the Swiss Tropical and Public Health Institute, University of Basel, Basel, Switzerland). 32The composition of the package of interventions that should be prioritized under the WHO Civil registration and vital statistics strategic implementation plan 2021-2025 will vary according to the needs and capacity of countries, but is likely to be similar for all countries in each of the categories that we have identified. 30he persistent application of these strategies is likely to bring about significant improvements in the availability and policy utility of mortality statistics.However, significant improvements in completeness of death registration may take some time, possibly a decade or longer in some countries.In the interim, established demographic methods applied to child survival, sibling survival and household death data available from surveys or censuses offer an alternative means for generating essential health intelligence on levels and patterns of mortality required to inform health planning. 33The policy utility of this information will be substantially enhanced by the application of automated verbal autopsy methods to a representative sample of deaths collected in surveys or censuses to provide cause of death data, as has been demonstrated elsewhere. 34ur study has some limitations, most notably the omission of recent vital statistics that are undoubtedly being produced in several low-and middleincome countries but are not readily accessible; as well as more disaggregated analysis of data quality at a subnational level where data are likely to be most relevant for planning.Estimates of death registration completeness are also subject to accuracy in the method used, either the empirical completeness method, or GBD or world population prospects estimated deaths.We used a standardized approach to estimate completeness rather than relying on country-derived estimates that would vary by country and lack suitability, which would reduce their intercountry comparability.We also did not assess completeness of death registration during the COVID-19 pandemic, in part because data for the pandemic years were not yet widely available, but also because of the abnormal registration environment that prevailed in most countries due to national responses to the pandemic, including movement restrictions.We were also unable to assess the role of public versus private facilities in reporting deaths.There is little available information on the relative completeness or accuracy of death data from different types of facilities, or even whether they were recorded in private or public institutions.
Vital statistics on deaths and causes of death are the cornerstone of any country's health information system, and the findings of this study should guide national and global action to improve them.Better data will allow countries to make better decisions leading to enhanced population health.■ Bull World Health Organ 2023;101:777-785| doi: http://dx.doi.org/10.2471/BLT.22

Table 1 . Criteria for the classification of mortality data from national vital registration systems Categories of mortality data % Registration completeness
a Of deaths reported for those younger than 80 years.Note: We classified countries where the most recent data were collected before 2015 as having no data available.

Table 2 . Completeness and quality of death registration and cause of death statistics by region; 2019 WHO Region % Deaths registered Deaths with medically certified cause of death Deaths with useable cause a
WHO: World Health Organization.a Includes deaths reported for those younger than 80 years.Bull World Health Organ 2023;101:777-785| doi: http://dx.doi.org/10.2471/BLT.22.289036Policy utility of routine mortality statistics Tim Adair et al.

Table 3 . Distribution of WHO Member States according to completeness, availability and quality of cause of death, 2019
Fig. 1.

Estimated completeness and quality of mortality and cause of death data for WHO Member States, 2019
WHO: World Health Organization.Bull World Health Organ 2023;101:777-785| doi: http://dx.doi.org/10.2471/BLT.22.289036Policy utility of routine mortality statistics Tim Adair et al.
.289036Policy utility of routine mortality statistics Tim Adair et al.